1. Field
The use of physiological methods to detect concealed information is as old as civilization. Even in ancient times observers noticed that whenever a crime suspect was being interrogated, the fear of possible exposure caused certain changes in his physiological functions. In ancient China a suspect was required to hold a quantity of rice in his mouth throughout the reading of his verdict. If the rice was still dry at the end, then the suspects guilt was considered proven since it was believed the fear of being exposed significantly reduced salivation in his mouth. Related techniques have been used throughout history by many very different peoples all over there world. However, modern “instrument-aided” physiological detection techniques did not arise until the early 20th century.
2. Prior Art
William Marston, an American psychologist and attorney, stands apart from the rest of the early research community studying instrument-aided “lie detection.” In 1913 the US National Research Committee formed a group of psychologists, including William Marston, to assess the potential use of known methods of lie detection for the needs of the counterintelligence service during World War I. Having conducted a substantial amount of research, the group concluded that the “blood pressure deception test” developed by Marston at Harvard University's psychology lab around 1913 was the most accurate method of all known at that time. Its accuracy in detecting lies was estimated to be as high as 97 percent. In 1923, John Marston, introduced for the first time, polygraph examination findings as proof of evidence in a court of law.
John Larson, a California police officer, designed the first practical prototype of a modern polygraph. After learning about Marston's “blood pressure deception test”, Larson developed an instrument capable of continuously recording blood pressure, pulse and respiration. Larson then set about developing an interviewing protocol, which was called the Relevant/Irrelevant (R/I) procedure. Throughout the interrogation, he would sprinkle questions relevant to the crime (“Do you own a 0.38 revolver?”) and questions that had nothing to do with it (“Are you twenty-eight years old?”). The assumption he made was that the innocent would have a similar physiological response to both types of questions, but guilty people would react more strongly to crime-relevant questions. The key problem with this approach was that even innocent people might be nervous, and crime-specific questions are generally fairly obvious. Nonetheless, a large number of criminal suspects were examined using this equipment, and highly accurate results were achieved.
Leonarde Keeler, working under Larson's guidance during the 1920s, played a crucial role in the deployment of the psycho-physiological method of lie detection in the United States. He came up with the first polygraph specifically designed to elicit hidden information (1933), the first guidebook on examination using a lie detector (1935), and founded a company for commercial production of these instruments. He also established the first training facility for polygraph examiners. Lastly, Mr. Keeler was the first to introduce polygraphy in the area of recruitment and crime prevention in business.
By the end of the 1930s three US companies were mass-producing lie detectors and marketing them throughout North America. Almost one hundred police departments in 28 states were making ample use of this instrument in their everyday work, and dozens of banks and commercial firms in the northern states introduced polygraphy for recruiting purposes and in-house investigations.
At about the time of the start of World War II the American Psychological Society undertook a special study to verify the reliability of polygraph examination in the interest of protecting the public. Having thoroughly analyzed the use of polygraph examinations in law-enforcement and the business environment, the research committee concluded that lie detection methods were sufficiently developed; the necessary technology existed; and a good number of well-trained specialists were available. Of these three factors, the last was considered most important. It was agreed that if a competent specialist was available, the examination results were quite useful. If such specialists were not available, it was concluded the method nor the equipment should be used.
At this point confidence in polygraphs started to increase and consequently their popularity did as well. Beginning in the early 1940s the polygraph was extensively used in protecting state secrets. Polygraphs were used in checking the personnel that had worked on the nuclear bomb project at the Oak Ridge Research Center.
Modern polygraphy consists of a computer system with bio-sensors. Sensors are used to measure and record a number of physiological changes that are related to the involuntary nervous system. The reliability of polygraphy is directly related to the number of measured and recorded inputs: typically the more inputs used, the more accurate the result. Decision-making is made by a trained expert human operator and is based on the aggregate of measurements taken, as well as individual characteristics.
Experts know that there is no direct connection between physical indicators and the sincerity of a person. Lie detectors record the level of involuntary nervous behavior of the examinee, but fail to identify the causative agent for changes measured by the instrument. The polygraph examiner has to make the final determination based on subjective input.
Functional Magnetic Resonance Imaging and Truth Detection
It has been known for over 100 years that changes in blood flow and blood oxygenation in the brain (collectively known as hemodynamics) are closely linked to neural activity. When nerve cells are active they consume oxygen carried by hemoglobin in red blood cells from local capillaries. The local response to this oxygen utilization is an increase in blood flow to regions of increased neural activity, occurring after a delay of approximately 1-5 seconds. This hemodynamic response rises to a peak over 4-5 seconds, before falling back to baseline (and typically undershooting slightly). This leads to local changes in the relative concentration of oxyhemoglobin and deoxyhemoglobin and changes in local cerebral blood volume in addition to change in local cerebral blood flow.
Hemoglobin is diamagnetic when oxygenated but paramagnetic when deoxygenated. The magnetic resonance (MR) signal of blood is therefore slightly different depending on the level of oxygenation. These differential signals can be detected using an appropriate MR pulse sequence, and manifest themselves as a blood-oxygen-level dependent (BOLD) contrast. Higher BOLD signal intensities arise from decreases in the concentration of deoxygenated hemoglobin since the blood magnetic susceptibility now more closely matches the tissue magnetic susceptibility. By collecting data in an MRI scanner with parameters sensitive to changes in magnetic susceptibility, one can assess changes in BOLD contrast. These changes can be either positive or negative depending upon the relative changes in both cerebral blood flow (CBF) and oxygen consumption. Increases in CBF that outstrip changes in oxygen consumption will lead to an increased BOLD signal. Conversely decreases in CBF that outstrip changes in oxygen consumption will cause a decreased BOLD intensity.
The precise relationship between neural signals and BOLD is still under active research. In general, changes in the BOLD signal are well correlated with changes in blood flow. Numerous studies during the past several decades have identified a coupling between blood flow and metabolic rate; that is, the blood supply is tightly regulated in space and time to provide the nutrients for brain metabolism. However, neuroscientists have been seeking a more direct relationship between the blood supply and the neural inputs/outputs that can be related to observable electrical activity and circuit models of brain function.
Although current data indicate that local field potentials, an index of integrated electrical activity, form a better correlation with blood flow than the spiking action potentials that are most directly associated with neural communication, no simple measure of electrical activity to date has provided an adequate correlation with metabolism and the blood supply across a wide dynamic range. Presumably, this reflects the complex nature of metabolic processes, which form a superset with regards to electrical activity. Some recent results have suggested that the increase in CBF following neural activity is not causally related to the metabolic demands of the brain region, but rather is driven by the presence of neurotransmitters, especially glutamate.
Some other recent results suggest that an initial small, negative dip before the main positive BOLD signal is more highly localized and also correlates with measured local decreases in tissue oxygen concentration (perhaps reflecting increased local metabolism during neuron activation). Use of this more localized negative BOLD signal has enabled imaging of human ocular dominance columns in primary visual cortex, with resolution of about 0.5 mm. One problem with this technique is that the early negative BOLD signal is small and can only be seen using larger scanners with magnetic fields of at least 3 Teslas. Further, the signal is much smaller than the normal BOLD signal, making extraction of the signal from noise that much more difficult. Also, this initial dip occurs within 1-2 seconds of stimulus initiation, which may not be captured when signals are recorded at long repetition (TR). If the TR is sufficiently low, increased speed of the cerebral blood flow response due to consumption of vasoactive drugs (such as caffeine) or natural differences in vascular responsivnesses may further obscure observation of the initial dip.
The BOLD signal is composed of CBF contributions from larger arteries and veins, smaller arterioles and venules, and capillaries. Experimental results indicate that the BOLD signal can be weighted to the smaller vessels, and hence closer to the active neurons, by using larger magnetic fields. For example, whereas about 70% of the BOLD signal arises from larger vessels in a 1.5 Tesla scanner, about 70% arises from smaller vessels in a 4 Tesla scanner. Furthermore, the size of the BOLD signal increases roughly as the square of the magnetic field strength. Hence there has been a push for larger field scanners to both improve localization and increase the signal. A few 7 Tesla commercial scanners have become operational, and experimental 8 and 9 Tesla scanners are under development.
BOLD effects are measured using rapid volumetric acquisition of images. Such images can be acquired with moderately good spatial and temporal resolution; images are usually taken every 1-4 seconds, and the sections in the resulting image typically represent cubes of tissue about 2-4 millimeters on each side in humans. Recent technical advancements, such as the use of high magnetic fields and advanced “multi-channel” RF reception, have advanced spatial resolution to the millimeter scale. Although responses to stimuli presented as close together as one or two seconds can be distinguished from one another, using a method known as event-related functional Magnetic Resonance Imaging (fMRI), the full time course of a BOLD response to a briefly presented stimulus lasts about 15 seconds for the robust positive response.
The science behind fMRI lie detection has matured with astonishing speed. One of the pioneers in the field is Daniel Langleben, a psychiatrist at the University of Pennsylvania. Langleben developed a hypothesis that in order to formulate a lie, the brain first had to stop itself from telling the truth and then generate the deception—a process that could be mapped with fMRI. By analyzing time-sequenced BOLD signal sources, fMRI reveals the pathways that thoughts have taken through the brain. Langleben concluded in 2002 in a paper published in the journal NeuroImage that there is “a neurophysiological difference between deception and truth” that can be detected with fMRI.
The subject took on a new urgency after 9/11 as security shot to the top of the national agenda. Despite questions about reliability, the use of polygraph machines grew rapidly, both domestically—where the device is employed to evaluate government workers for security clearances—and in places like Iraq and Afghanistan, where polygraphers are deployed to extract confessions, check claims about weapons of mass destruction, confirm the loyalty of coalition officers, and grill spies. The Department of Defense Polygraph Institute (DoDPI) put out a call for funding requests to scientists investigating lie detection. Grants from DoDPI, the Department of Homeland Security, DARPA, and other agencies triggered a wave of research into new lie-detection technologies. Since the events of 9/11, there are now over 50 labs in the US alone doing this kind of research.
Langleben's team, whose work was funded partially by DARPA, began to focus on eliminating one major source of polygraph error—the subjectivity of the human examiner. Langleben and his colleagues developed pattern-recognition algorithms that identify deception in individual subjects by comparing their brain scans with those in a database of known liars. In 2005, both Langleben's lab and a DoDPI-funded team led by Andrew Kozel at the Medical University of South Carolina announced that their algorithms had been able to reliably identify lies.
Today's fMRI scanners are bulky, cost up to $3 million each, and in effect require consent because of their sensitivity to head movement. This technology is not considered applicable to individuals who might want to keep information concealed, and in spite of the advances made by Langleben in automating the detection of lies through sophisticated computer-based algorithms, the system still requires trained and skilled operators.
Evoked Potentials
In neurophysiology, an evoked potential is an electrical potential recorded from a human or animal (or “biopotential”) following the presentation of a stimulus, as distinct from spontaneous potentials such as electroencephalograms or electromyograms. Evoked potential amplitudes tend to be low, ranging from less than a microvolt to several microvolts, compared to tens of microvolts for EEG, millivolts for EMG, and often close to a volt for EKG. To resolve these low-amplitude potentials against the background of ongoing EEG, EKG, EMG and other biological signals and ambient noise, signal averaging is usually required. The signal is time-synchronized to the stimulus and since most of the noise occurs randomly, the noise is averaged out.
Signals can be recorded from the cerebral cortex, brainstem, spinal cord and peripheral nerves. Usually the term “evoked potential” is reserved for responses involving either recording from, or stimulation of, central nervous system structures. Thus evoked CMAP (compound motor action potentials) or SNAP (sensory nerve action potentials) as used in NCV (nerve conduction studies) are generally not thought of as evoked potentials, though they do meet the above definition.
Sensory evoked potentials (SEP or SSEP) are recorded from the central nervous system following stimulation of sense organs (for example, visual evoked potentials elicited by a flashing light or changing pattern on a monitor; auditory evoked potentials by a click or tone stimulus presented through earphones) or by electrical stimulation of a sensory or mixed nerve. They have been widely used in clinical diagnostic medicine since the 1970s, and also in intraoperative neurophysiology monitoring (IONM), also known as surgical neurophysiology. There are three kinds of evoked potentials in widespread clinical use since the 1970s: auditory evoked potentials, usually recorded from the scalp but originating at brainstem level; visual evoked potentials, and somatosensory evoked potentials, which are elicited by electrical stimulation of peripheral nerves.
To measure evoked potentials electrodes need to be attached to various points of on the scalp. Typically, the head is measured using a standardized EEG measurement technique to determine the optimal locations (each location corresponding to a type of EP that will be measured—e.g. the two locations on the back of the skull for the visual cortex, etc.), which are typically marked with magic marker. Each of these spots is rubbed with an oil-removing scrub to get rid of the skin oil. Then an electrode dipped in a liberal quantity of conductive gel is applied to each location, and affixed with a strip of adhesive tape.
For visual evoked potentials (VEP), the subject is placed in front of a computer screen, which shows a pattern of white and black squares like a chessboard, and a red dot in the middle that the subject is supposed to focus his/her eyes on with minimal movement. The procedure is done one eye at a time, with the eye that is not being tested blocked off with an eye patch. During the actual procedure, these squares alternate (white ones become black, black ones become white) at a rate of several times a second. This produces responses in the visual cortex that are picked up by the electrodes. Since the computer controls the exact timing of the changes of the square colors, and receives the electrical response in the corresponding electrodes, it is able to determine precisely the amount of time it takes for the visual stimulus to reach the visual cortex. For the somatosensory evoked potentials (SEP), additional electrodes are applied, in the same manner as described earlier.
There are many things going on at once in the brain, so it is difficult to determine when the evoked potential from a particular stimulus arrives from just one stimulus. A common technique used to amplify the signal is called ensemble averaging. The stimulus in each evoked potential experiment is presented multiple times, and since other signal components besides the evoked potential are not related to the signal, the computer can discriminate and amplify the one consistent peak or series of peaks that are caused by the applied stimulus.
In the 1980's Towle, Heuer & Donchin demonstrated that a subject would produce a positive signal peak approximately 300 msec after onset of stimulus (P-300) in response to visual stimuli that consisted of two sets of photographs, one of generally known politicians, the other of generally known movie stars. The subjects were instructed to count one or the other category. Each time an image from the task-relevant class was displayed, the subject produced the traditional P-300 response. This research confirmed what was well known since the mid 1960's, primarily that P-300's are elicited by stimuli that provide information necessary for the performance of an explicit task assigned by the experimenter—such as counting movie stars. It was also accepted that P-300's would be not be present in the absence of such an explicit task.
In the late 1990's early 2000's Farwell et. al. hypothesized that stimuli that are not explicitly task-relevant would still nevertheless elicit a P-300 if they are particularly significant to the subject due to his/her past knowledge of the subject matter of the stimuli. This theory is based on the “Context Updating Model”. This model is based on the idea that when a stimulus that is significant for the subject is provided, he or she can be expected to take particular note of it, thus revising his/her internal representation of the current environment and generating a P-300.
The Context Updating Model then provides an alternative means to ascertaining concealed knowledge of a human subject. In contrast to focusing on physiological changes elicited through an interrogative process, a new technique for verifying knowledge can be created by measuring the response to known-relevant stimuli which serve as a proxy for said knowledge. It is logical to assume that a human subject will produce one kind of signal in response to images of people, places and things for which he/she has knowledge and another kind of signal in response to images of people, places and things for which he/she has no knowledge.
There exist many methods using physiological metrics for the detection of concealed information. The prior art is replete with examples of various means and methods for ascertaining concealed knowledge of a human subject. Functional MRI has created a window to the mind permitting scientists to observe the areas of the brain and the neural pathways involved in lying. Many attempts, such as those disclosed by Farwell, utilize biopotentials including EEG in combination with an interrogative process to determine if information provided by a human user is truthful, or if it is part of a subterfuge. Almost every example of “Truth Detection” in the prior art have two important limitations that are overcome by the present invention: (1) The process involves an auditory or visual interrogation of the human subject, and (2) The system requires a skilled operator to administer the test and/or provide interpretation of the results. This is the case with modern polygraphy, functional MRI and other physiologically-based technologies.
Many inventors have devised myriad of approaches attempting to provide inexpensive, minimally invasive, and rapid knowledge verification systems which could detect concealed knowledge (typically guilty knowledge) of human users. However, none have succeeded in producing a system that is practical and desirable for use in applications where either no direct interrogation is desired, or no trained operator is available to administer the examination or provide expert analysis of the results. Because of these and other significant limitations, commercially viable automated knowledge verification systems have not yet come to market.
The present invention overcomes all of the aforesaid limitations by combining a system in which there is (1) no interrogation and (2) no subjective human-expert analysis with robust hardware elements such as a simple headband, a rugged biopotential amplifier, a fast microprocessor and a user-friendly automated software interface. The present invention utilizes an electrode-studded headband assembly that correctly positions an array of disposable Ag/Ag—Cl electrodes in the desired anatomical position along the Z-axis of the head of the subject. To apply, the operator places a dollop of conductive gel on the surface of each electrode, then lowers the band over and onto the subject's head and adjusts the tightness using Velcro straps. The leads from the headband are subsequently connected to the biopotential amplifier which is electrically attached to a computer such as a PC or laptop. In one potential embodiment, with the subject seated before the video monitor (which provides the stimulating visual images), the operator starts the control/analysis software, performs some simple calibration and test routines and begins the examination. During the examination, the software resident on the PC or laptop controls all aspects of the system's operation. At the end of the test, information relevant to the examination is provided to the operator in both electronic and hard-copy format.
This novel method of utilizing biopotentials coupled to the easy-to-use automated control/analysis software overcome many significant limitations of the prior art. Subjectivity and interpretation is not required with the present invention. Instead, the human-expert analysis of the prior art inventions is replaced by an objective software analysis algorithm. The only training required by the present invention is in its setup and operation—training that can be accomplished quickly and inexpensively.
It is an object of the present invention to overcome the problems, obstacles and deficiencies of the prior art.
It is also an object of the present invention to provide an improved system for verifying concealed knowledge of a human subject utilizing biopotentials and physiological metrics. It is further an object of the present invention to provide an improved system and method for enhancing the training of human subjects.
Accordingly, one embodiment of the present invention is directed to a system and method for verifying human knowledge by means of measuring subject's physiological responses to input stimuli comprising: (a) a stimuli exposure system that exposes the subject to input stimuli; (b) a sensor system that monitors and records subject's physiological responses before, during, and after input stimulus is presented to subject; (c) a computer system comprising processing and algorithmic elements that determines if subject's physiological responses indicate subject possesses knowledge of interest; and (d) a reporting system that presents a report to the operator which indicates: (i) input stimuli exposed to subject, (ii) physiological responses of the subject before, during, and after each input stimulus is exposed to subject, and (iii) determination of the computer system of whether subject possesses knowledge of interest.
The stimuli exposure system of this first embodiment includes Probe, Relevant and Gallery data; and a visual display comprising an LCD video monitor. A Protocol Creation Algorithm presents the probe, relevant and gallery visual stimuli in a weighted pseudo-random sequence. The Probe image data are not generally known to human subjects but relevant to the knowledge to be verified. The relevant image data are generally known to human subjects but not relevant to the knowledge to be verified. The gallery image data are not generally known to human subjects and not relevant to the knowledge to be verified.
Although exclusively visual stimuli are described in the first embodiment, alternative stimuli such as auditory, tactile, and olfactory stimuli may also be presented in other embodiments. These may be presented independently or in combination. The stimuli exposure system presents auditory stimuli through recorded audio playback. Tactile and olfactory stimuli are presented through physical samples.
The sensor system of this first embodiment includes a removable physiological data amplifier connected to a human subject via disposable Ag/Ag—Cl electrodes and an analog-to-digital (A/D) converter to digitize physiological data for subsequent storage on the computer system. Although this first embodiment reads specifically biopotential signals, additional physiological signals may also be monitored. These include functional magnetic resonance images or positron emission tomography. The data gathering would require the use of a Functional Magnetic Resonance Imaging (fMRI) machine or Positron Emission Tomography (PET) scanner.
The computer system of this first embodiment includes data analysis software. Analysis software discriminates subject's event-related response from exogenous stimuli by means of neural network analysis, parametric analysis, statistical analysis, pattern matching, or wavelet processing to provide an output of verification or non-verification of knowledge of interest. The reporting system presents the results of the determination of the data analysis software.
When an image is recognized, the user would be expected to produce a P-300 response indicative of said recognition. A predetermined number of instantiations of the Probe, Relevant and Gallery images would be ensemble averaged to ameliorate the contributions of artifacts. By examining the ratio of recognized-to-nonrecognized Probe and Gallery images, an objective metric for relevant knowledge verification could be realized. Statistical evaluation of the false-positives (incorrectly recognizing a Gallery image as a Probe image) and false-negatives (failing to correctly recognize a Probe image) for a specific subject would provide a determination with respect to subject's relevant knowledge. Relevant images provide benchmark measurements for recognized non-relevant images.
A second embodiment of the present invention is directed to a system and method for enhancing the training of human subjects in which subject's successful assimilation of information provided at a prior training event is verified by means of measuring subject's physiological responses to input stimuli. This second embodiment is comprised of the method and apparatus of the first embodiment in addition to a method wherein the results of the verification of subject's successful assimilation of information provided at a prior training event are used to determine if subject should repeat the training event. In addition, these results can be used to improve future training events.
With respect to this method, subject would complete a training exercise and subsequently be tested to verify relevant knowledge and determine retention. This test could be conducted using objective metrics. An electronic hand-operated switch could be optionally provided to the user to depress each time an image is recognized. Alternatively, subject could be asked to perform an abstract cognitive task such as a visualization task or counting task when an image is recognized. These results can further be utilized in modifying the training curriculum or training technique to optimize the exercise. If a particular set of information was not being taught well, the results of the test would show a consistent lack of knowledge in that area. The teaching methods for that area could then be changed in future training.
Other objects and advantages will be readily apparent to those of ordinary skill in the art upon viewing the drawings and reading the detailed description hereinafter.